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ABSTBACT 

The  objective  of  this  study  is  to  develop  and  test  two 
new  approximate  solution  techniques  for  a  moving  target 
problem  in  discrete  time  and  space  where  both  the  searcher 
and  the  target  have  constraints  on  their  paths.  The  first 
technique  is  an  application  of  the  Local  Search  Method  and 
the  second  technique  is  an  application  of  the  FranK-Wolfe 
Method.  The  motivation  for  looking  at  approximate  methods 
is  that  the  problem  is  NP-ccmplete  and  optimal  solution 
techniques  become  impractical  for  large  size  problems. 
Experiments  showed  that  the  Local  Search  Method  approach  is 
an  efficient  technique  for  obtaining  approximate  solutions. 
However,  the  Frank-Wolfe  Method  approach  does  not  perform 
well    for   the    problem. 
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I.    INTBOPOCTION 

A.  TBE    PRO BLEB 

The  problem  considered  here  is  the  search  for  a  moving 
target  in  discrete  time  and  space,  where  there  are 
constraints      on  the      searcher's      movement.  The   target     is 

assumed  to  move  among  a  finite  set  of  cells  C={1,-.-,N} 
according  to  a  specified  Markov  transition  matrix 
Z=£z(i,j)}.  The   target's      motion      is      independent  of      the 

searcher's  actions.  The  transition  matrix  is  known  to  the 
searcher  as  well  as  the  initial  distribution  of  the  target 
over    the      search    cells.  In   each    time      period    one      cell   is 

searched.  The  searcher  is  atle  to  move  from  his  current 
cell,  say  i,  to  a  cell  which  must  be  selected  frcm  a  given 
subset  C  (i)  of  the  set  C.  Thus,  the  search  cell  in  a  given 
time  period  must  be  within  some  specified  neighborhood  of 
the      search      cell   in      the      previous      time    period-  If      the 

searcher  and  the  target  are  in  the  same  cell  i,  the  target 
will  he  detected  with  probability  g(i).  If  the  target  is 
not  in  the  cell  which  is  searched,  it  cannot  be  detected 
during  the  time  period.  The  objective  of  tie  search  is  to 
find  a  T-time  period  search  path  which  minimizes  the  prob- 
ability   of    non-detection    (PND)  . 

B.  BACKGROUND 

The  problem  of  searching  fcr  a  moving  target  in  discrete 
time  and  space  has  received  considerable  attention.  The 
problem  is  difficult  because  of  its  complexity.  Trumael  and 
Weisinger  [Ref.  1]  proved  that  optimal  constrained  path 
search  problem  for  a  stationary  target  in  discrete  tisne  and 
space  where   the  time  horizon  is   finite  and  the   measure  of 


effectiveness  is  PND,  is  NP-Complete.  Since  the  stationary 
target  problem  is  NP-Complete,  the  moving  target  problem  is 
also  NP-Complete.  For  more  information  on  NP-complete  prob- 
lems, see  [Ref.  2].  The  optimal  solution  techniques  which 
have  been  used  to  solve  the  constrained  path  moving  target 
problem   are: 

1.  lotal    enumeration 

2.  3he  dynamic    programming   technique   of    Eagle   [Ref.    3]. 

The  total  enumeration  method  is  complete  enumeration  of 
all  possible  search  paths.  The  disadvantage  ox  this  metnod 
is  the  exponential  increase  in  computer  time  as  the  problem 
size  increases.  However,  it  has  the  advantage  of  requiring 
very  little  computer  storage.  Eagle  formulates  the  problem 
as  a  Partially  Observable  Markov  Decision  Process  {POMDPj  , 
and  the  method  requires  extensive  computer  storage,  bit 
finds  solutions  more  quickly  than  the  total  enumeration 
methcd. 

Since  the  problem  is  NP-Complete,  it  is  difficult  to 
solve  large  problems.  For  large  problems,  heuristic  or 
approximate  solution  techniques  have  oeen  proposed.  Such 
methods  were  developed  by  Stewart  [Ref-  4]  and  Eagle 
[Ref.    5]. 

Stewart  developed  a  branch-and-bound  algorithm  tor  the 
solution    of      the    problem.  However,      optiaality      cannot   be 

guaranteed  because  exact  bcunds  cannot  be  obtained. 
Never tniess,  Stewart  used  one-dimensional  a  random  walk  as  a 
test  problem  and  demonstrated  that  the  algorithm  performs 
efficiently. 

Eagle* s  approximate  procedure  is  a  simpler  version  of 
his  optimal  dynamic  programming  method.  Eagle  used  moving 
horizcn  policies  and  also  looked  for  a  lower  bound  for  the 
minimal        PND.  Eagle's        computational      experience        with 


m 


two-dimensional  search  problems  showed  the  m-time  period 
moving  horizon  policies  (m-TPMH)  do  not  necessarily  perfor 
better  as  m  increases-  Nonetheless,  the  method  does  show 
some  promise.  The  m-TPKH  policy  selects  as  the  next  search 
that  cell  wnich  would  be  optimal  if  m-time  periods  remained 
in    the    problem. 

fiecently  Eagle  and  Yee  developed  a  new  approximate  solu- 
tion method  by  using  the  convex  simplex  method  [fief.  6]« 
Two-dimensional  computational  experiments  showed  that  the 
method   performs    well. 

In  this  thesis  two  new  approximate  solution  techniques 
will  te  studied  and  tested.  The  first  method  is  a  heuristic 
approach  which  is  an  application  of  the  Local  (Neighborhood) 
Search.  The  second      technique  is      an      application   of      the 

Frank-Wclfe   method. 
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II.  FOBflDIATION  OF  THE  PROBLEM 

In  this  chapter   we   will  formulate   the   problem  as   a 
nonlirear  program. 

A.   DEFINITION  OF  SYMBOLS 

T  :  Number  of  time  periods. 

N  z   -Number  of  cells. 

q  (i)  :  Prob  {detecting  the  target  J  target  and  searcner 
ar€  in  cell  i}  . 

Z  =  {z(i,j)}  is  an  NxN   Markov  transition  matrix  where  the 

(i,j)th  element  is  the  probability  that  the  target  in  cell 

i  at  time  period  t  will  move  to  cell  j  at  time  period  t+1. 

x  (t)    =  {x  (1«t) ........ , x  (N  ,t) }   represents  the   searcher 

Srobability  distribution  at  time  t.  That  is,  x(i,t) 
enctes  the  probability  of  the  searcher  being  in  cell  i  at 
time  period  t.  At  the  start  of  the  search,  the  searcher 
cell  is  assumed  to  be  known  with  certainity,  i.e.,  x(1) 
consists  of  1  one  and  all  others  zero. 

P(t)  =  {P(1,t),.. .p(N,t)}  represents  the  target  prob- 
ability distribution  at  time  t.  That  is.  p{i,t)  is  the 
probability  target  is  in  cell  i  at  the  begming  of  time 
period  t  without  being  detected  by  the  previous  t- "J 
searches.  The  initial  target  distribution  P{1)  is  assumed 
to  be  known  by  the  searcher. 

R(ifj,t)  is  the  probability  that  the  target  is  in  cell  i 
without  being  detected  by  the  previous  t-1  searches  and 
the  searcher  is  in  cell  j. 

S(t)  =  {S(i,j#t)l  is  an  NxN  Markov  transition  matrix  for 
the  searcher.  The  (i.j)th  element  is  zero  if  subset  C(i) 
does  not  include  j  (patn  constraints) - 

S  =  (S(1)  .. ..  .  ,S  j[T-1)  }  is  called  a  search  plan  which 
includes  all  possible  searcher  motions  for  T-time  perioi, 
given  that  the  searcher  starting  cell  is  known. 


B.   NIP  FORMULATION  OF  THE  PROBLEM 

New  we  neea  to  compute  x  (t)  *s  and  R(i,j,t)'s.  Since 
x(1)  is  known,  we  can  determine  x(t)  by  the  Markovian  motion 
as  shewn  below: 


x(t)  =  x(t-1)  S<t-1),   t=2,..-,T. 

1  1 


The  R<i,j,t)  consists  of  two  Markov  processes.  Note  that 
R(i,j,1)  is  assumed  to  be  known-  So,  the  R(i,j,t)  can  be 
computed  as  follows: 


R(i.j,t)    =    {1-q  (i)d(i,j)J  2_    B(k,lrt-1)z(k,i)S{l.j,t-1) 

i, j,k, 1-1, 2, ......  N      and   t=2#3#..-.rT 


where  d{i,j)  is  one  if  i  eguals  j,  zero  otherwise. 
Therefore,  given  P(1)#  g(i)#  S,  and  Z,  the  T-time  prob- 
ability   of   nondetection   is 

PND   =J"""r(j,j#T),         i,j=1,2,--.,N. 

We  want  to  fina  the  search  plan  S  which  minimizes  the 
T-time  period  PND,  subject  to  the  path  constraints.  The 
final  NLP  form  of  the  problem  is  given  in  Table  1.  The 
foliowing  properties  of  the  NLP  were  proven  by  Eagle  and  Yee 
[Ref.    6  ]- 

Proposition  1:  The  minimum  of  the  hLP  is  achieved  by  a 
deter ttinistic      search    plan.,  A      deterministic    search      plan 

consists   of   ones    and    zeros. 

Proposition  2:  S  is  a  deterministic  search  plan  if  and 
only    if    S   is   ac    extreme   point    of   the    linear    constraints. 

Propositioc  3:  The  objective  function  is  linear  in  S 
wnen    constrained    to    any   edge   of    the    simplex. 
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1        -.-..... 

TABLE 
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THE    PflOBLEM 

mia      PND 

sutject   to: 

S(t)     1    =    1 
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t=1,...,T-1 

S(t)    >    0 

t 

t=  1  ,  .  .  .  ,  T-  1 

S(i,j,t)    =    0 

t 

t=1,.-.,T-1 
i=1,...,N 

.. 

j  $.    C(i) 
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Ill-  UJ  LOCAL  SEABCH  ALGOBJTHM 

The  optimal  solution  technigues  for  hard  combinatorial 
optimization  problems  require  more  computer  time  when  the 
size  of  the  problem  increases.  Therefore,  they  become 
difficult  to  implement  for  large  size  problems-  One  of  the 
most  succesful  methods  of  attacking  them  is  the  Local  Search 
or  Neighborhood  Search.  However  unsatisfying  mathemati- 
cally, this  approach  is  certainly  valid  in  practical  situ- 
ations. The  main  shortcoming  of  the  method  is  that  there  is 
no  criterion  to  tell  whether  the  obtained  solution  is  a 
global  optimum.  The  local  search  method  is  most  applicable 
for  optimization  in  two  main  types  of  problems: 

1.  When  the  analytic  relationship  of  the  independent 
variables  and  the  objective  function  is  not  known, 
but  the  value  of  the  objective  function  can  be  evalu- 
ated at  individual  points  by  experiments. 

2.  When  the  analytic  form  of  the  objective  function  is 
known  but  there  is  no  finite  algorithm  for  obtaining 
the  extreme  value  (s)  in  a  closed  form. 

The  efficiency  of  the  method,  relative  to  another,  is 
defined  in  terms  of  a  suitable  cost  function  such  as  number 
of  points  or  experiments  required  for  localizing  the  optimum 
witnic  a  specified  range.  One  of  the  best  known  illustra- 
tions of  this  technique  is  the  traveling  salesman  problem 
(TS?)  where  the  objective  is  to  find  the  shortsst  path 
passing  through  each  of  a  number  of  points  exactly  once  and 
then  returning  to  the  starting  point.  For  other  examples 
refer  to  [  Ref .  7  ]«. 

Since  the  problem  presented  in  the  previous  cnapter  is 
NP-conplete  and  the  current  optimal  solution  techniques  are 
impractical  for  large  size  problems,  we  can  try  to  apply  the 
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local   search  method   as  a   heuristic   approach   for   good 
approximate  solutions. 

A.   2  BE  ALGORITHM 

First,  we  will  define  some  concepts  in  order  to  intro- 
duce general  local  search  algorithms  and  then  apply  this 
method  to  the  constrained  path  moving  target  search  problem. 

Definition  1:  An  instance  of  an  optimization  problem  is  a 
pair  (F,c)  where  F  is  any  set,  the  domain  of  feasible 
points,  and  c  is  the  objective  function,  a  mapping 

c:  F — >  Rl 

Definition  2:  Given  an  optimization  problem  with  instances 
(F,c)  ,  a  neighborhood  is  a  mapping 

N:  F — >  2F 

defined  for  each  instance. 

The  general  local  search  algorithm  is  described  as 
follows:  Given  an  instance  (F,c)  of  an  optimization  problem 
(minijiizing)  and  an  initial  feasible  point  s,  we  search  the 
neighborhood  N  (s)  for  an  m£N(s)  with  c  (m)  <  c{s)-  So,  we 
start  with  an  initial  feasible  point  s  and  attempt,  to 
improve  the  objective  function  by  searching  the  neighborhood 
of  s.  If  an  improvement  is  found,  another  search  is  made  in 
the  neighborhood  of  the  improved  point-  This  continues 
until  nc  improvement  can  be  made. 

Since  we  know  that  the  solution  of  tne  NLP  is  an  deter- 
ministic search  plan  (Proposition  1)  and  it  is  assumed  that 
the  searcher  starting  cell  is  known,  we  can  reduce  our 
feasible  region  to  search  paths  s=  {s  ( 1) ,- . . , s  (T) } ,  where 
s (t )  is  tne  searcner  cell  at  time  period  t.  Therefore,  we 
can  start  with  an  initial  feasible  search  path  s,  an  objec- 
tive function  value  END  (s)  ,  and  attempt  to  find  a  search 
path  m  in  a  neighborhood  of  s  such  that  PND(m)  <  PND  (s)  .  we 
define  the  k-change  neighborhood  of  search  patn  s  as  all 
feasible  search  paths  m  which  are  identical  to  s  except  in  k 
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consecutive   time    periods.      So,    the   algorithm  for   the    proDlem 
becomes : 

Step    1.    Start  with    an  initial   feasible   search    path   s. 

Step  2.  Find  the  all  possible  k-change  neighborhood  paths 
by  removing  k  consecutive  cells  from  s  (keeping  the  others 
unchanged)  then  replacing  them  with  k  new  cells  satisfying 
the  path  constraints- 
Step  3.  Find  the  k-change  neighborhood  path  m  with  the 
minimum  PND  (m)  - 

Step    4.      If    PND  (m)         <    PND(s),      set   s=m   and   go      to   step   2. 
Otherwise,    STOP;   current   s  is  k-optimal. 

The  most  difficult  task  in  any  application  is  to  decide 
of  the  proper  value  of  k  for  the  k-change  policy.  Selecting 
the  rignt  k  leads  to  very  effective  heuristics  for  the  TSP 
[Eef.  7]-  We  will  test  the  present  algorithm  for  k=1,2  and 
3. 


B.       INITIAL    FEASIBLE    SEARCH    PATH 

The  algorithm  requires  two  stages.  First,  determine  the 
initial  feasible  starting  search  path  and  then  apply  tiie 
procedure  to  it.  The  initial  feasible  point  may  be  chosen 
randomly  or  it  may  be  carefully  constructed.  In  practice  we 
need  a  guick  and  a  good  approximate  solution.  We  do  not 
want  to  be  far  away  from  the  optimal  when  we  start.  In  this 
study  we  will  use  the  modified  myopic  policy  (rtMP)  to  deter- 
mine an  initial  feasible  starting  path.  The  myopic  policy, 
which  gives  optimal  solution  for  a  stationary  target  prob- 
lems, always  searches  in  the  next  time  period  t,  the  cell  i 
which  has  the  largest  probability  of  detecting  the  target, 
{q (i) p  (i,t) } .      The    M  MP   is   defined   as    follows. 

Fcr  the  next  time  period  t,  searcher  moves  from  his 
currect  cell  i  to  that  accessible  cell  j  which  has  tne 
largest  q(j)p(j,t)  value.  If  all  accessible  cells  j  have 
identical  values  of  g(j)p(j,t),  then  the  searcher  will  move 
to      the    accessible      cell      which      has   the      minimum      Euclidian 
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distance  to  the  cell  k  having  the  largest  g(k)p{k,t)  value. 
If  the  Euclidian  distances  are  equal,  these  ties  are  broken 
randomly.  There  is  probably  a  smarter  technique  for 
breaking  ties. 

Ihe  MMP  is  a  heuristic  construction  meant  to  produce  a 
good  starting  path,  because  the  neighborhood  k-change  proce- 
dure may  not  be  very  powerful,  We  will  also  test  the 
(k- 1) -optimal  solution  as  a  starting  path  for  the  k-change 
policy. 
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IV.  COMPOTATIONAL  EXPERIENCE 

Our  computational  experience  is  based  on  two-dimensional 
search  problems.  All  experiments  presented  in  this  chapter 
were  performed  on  an  IBM  3033  mainframe  computer  using 
Fortran  77.  The  Local  Search  Algorithm  was  used  to  solve 
three  10-time  period  search  problems,  where  N=9 ,  25  and  49. 
For  each  problem,  values  of  k  =  1,  2  and  3  were  used.  The 
constraints  on  the  searcher  and  the  target  paths  are  defined 
as;  the  searcher  (target)  is  able  to  move  to  the  cell  previ- 
ously searched  (occupied)  plus  all  adjacent  cells.  Cells 
are  adjacent  if  they  share  a  common  side.  For  example,  in 
Figure  4.1  the  adjacent  cells  of  cell  1,  4  and  5  are: 

C(1)={1,2,4)  ,  C(4)={1,4,5,7}  ,  C  (5)  =  {2,  4  .  5,6  ,8} 

The  target  transition  matrix  is;  the  target  remains  in  the 
previously  occupied  cell  with  probability  0-4,  and  moves  to 
the  adjacent  cells  with  probability  0.6/m  where  m  is  the 
number  of  adjacent  cells.  The  probability  of  detection  is 
1.0    fcr    each    cell,    i.e.,    g(i)  =  1.0      ,i-1,...,N. 

A.       3X3      PROBLEH 


1 

2 

3 

4 

5 

6 

7 



8 

9 
.  

Figure  4.1    9  Cell  Search  Grid. 
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Id  this  problem  the  target  moves  among  the  9  cells  shown 
in  Figure  4-1.  The  searcher  starts  in  cell  1  and  the  target 
in  cell  9,  i.e. ,  initial  target  distribution  is  that  the 
target  is  in  cell  9  with  probability  1.0  and  in  the  other 
cells  with  probabiliy  0-  The  10-tiae  period  search  results 
are  shown  in  Table  2.  An  optimal  solution  was  obtained  for 
all  k=1 ,  2,  3.  However  CPU  time  increases  exponentially  as 
k  increases.  As  can  be  seen,  the  MM?  solution  is  very  close 
to  the  optimal.  The  Fortran  code  for  this  problem  is  given 
in  Appendix  A. 


1 

TABLE 
3x3    PROBLEM 

2 
EESULTS 

Search    Path 

PND 

CPU 
Time  [sec) 

Myopic 

4 

5 

8    5    6    5    8    5 

6 

5           0.2328 

k   =    1 

4 

5 

8    9    6    5    8    5 

6 

5           0.2214 

0.51 

k   =   2 

2 

5 

8    9    6    5   8    5 

6 

5           0.2214 

3.  10 

k    =  3 

2 

5 

8    9    6    5   8    5 

6 

5           0.2214 

26.18 

OPTIMAL    PND 

= 

0.2214 

B-       5X5    PROBLEM 

In  this  example  the  target  and  the  searcher  moves  among 
25  cells  (Figure  4.2).  The  searcher  starts  in  cell  1  and 
the  target  starts  in  cell  13  with  certainty.  The  10-time 
period    search    results   are   shown   in    Table    3. 

Vie  found  an  optimal  solution  only  for  k=3,  but  solutions 
for    k=1 ,         2   are      close    to    the      optimal.         The      k=2    solution 
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Figure    4.2        25  Cell   Search   Grid. 


TABLE   3 
5x5    PROBLEM    BESULTS 

Search   Path 

Myopic         6    7     12    13    14    13    18  13    14    13 

k    =    1  6    7     12    13    14    19    18  13      8       9 

k    =    2  2    7     12     13       8    13    18  19    14       9 

k    =    3  67       813    12    1718  19    14       9 

OPTIMAL    PND    =  0.4886 


PND 

CPU 
Time  (sec) 

0.5099 

0.4907 

1.09 

0.4887 

10.09 

0.488c 

75-23 

differs  from  tne  optimal  solution  only  in  the  fourth  digit- 
Similar  to  the  3x3  problem  the  MMP  solution  is  close  to  the 
optimal    and   CPU   time    increases    exponer. tially. 

Co        7X7      PROBLEM 

In  this  proolea  the  target  and  the  searcher  moves  among 
49  search  cells  shown  in  Figure  4.3.  The  searcher  starts  in 
cell    1    and    the    target    starts   in    cell    17.       The    10-time    period 
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search  results  are  shown  in  Table  4.  He  obtained  the 
optimal  solution  for  only  k=3.  The  percentage  difterence 
between  the  MMP  solution  and  the  optimal  solution  is  only 
1.2  9L  For  k=1  and  2  same  result  Ktia  obtained. 
Exponential  increase  in  CPU  time  is  more  steep  than  the 
previous  problems. 

In  addition,  the  total  enumeration  method  was  used  to 
find  all  optimal  paths  for  each  problem.  The  results  are 
shown  in  Table  4. 
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26 
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31 
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45 

46 

47 

48 

1 

49 

Figure  4.3   49  Cell  Search  Grid- 

In  ract,  for  the  5x5  and  7x7  problems  we  increasea  the 
search  area  of  the  3x3  problem  with  toe  saic  target  and 
searcher  starting  cells  and  the  target  transition  matrix. 
For  each  pronlein  a  similar  pattern  was  observed  tor  optimal 
paths.  Specificall y ,  the  searcher  goes  to  the  target's 
starting  cell  as  soon  as  possible,  tnen  starts  to  search 
outside  of  this  cell  similar  to  the  expanding  sguare  search. 
Graphical  representation  of  exponential  increase  in  CP'i  time 
as    k    increases,    is   shewn    in    Figure    5. 1- 
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TABLE    4 
7x7    PfiOBLEU    BESULTS 


Search    Path 

Myopic        2   9     10     17   24    17    16    17    10  11 

k    =    1            2    9    10     17    24    23    16    17    18  11 

k   =    2           2    9     10    17   24    23    16    17    18  11 

k   =    3           2    9    16    17   24    17    18    11    10  9 
OPTIHAL    PND    =    0-5005 


PND 

CPU 
Time  (sec) 

0.5066 

0.5063 

1.83 

0.5063 

9-55 

0.5005 

117.73 

TOTAL    ENUMERATION    RESULTS    1 


Prcblem      Possible    Optimal    Paths 


3x3 


2 
2 

2 
4 
4 

4 


Optimal        CPU 
PND         Time (sec) 


0.2214 


240 


6    11     12    13      8       9    14    19    18    17 


5x5 


b 

6 
b 

6 
6 
2 
2 
2 
2 
2 
2 


11 
7 

7 
7 
7 

7 
7 

7 
7 
3 
3 


12 

8 

8 

12 

12 

12 

12 

8 

8 

6 

8 


13 
13 
13 
13 
13 
13 
13 
13 
13 
13 
13 


18 
12 

14 
8 
18 
8 
18 
12 
14 
12 
14 


9 
17 
19 

9 
19 

9 
19 
17 
19 
17 
19 


14 
16 

18 
14 
14 
14 

14 
18 
18 

18 
18 


9 
19 
17 
19 

9 
19 

9 
19 
17 
19 
17 


8 
14 
12 
18 

6 
18 

6 
14 
12 
14 
12 


7 

9 
7 
7 
7 
7 
7 
9 
7 
9 
7 


0.4886       1656 


7x7 


o 

8 
8 
2 
2 
2 


9 
9 
15 
3 
9 
9 


10 
16 
16 
10 
10 
16 


17 
17 
17 
17 
17 
17 


18 
24 
24 
18 
18 
24 


17 
17 
17 
17 
17 
17 


24 
18 
18 
24 
24 
18 


23 
11 
11 

23 
23 
11 


16 
10 
10 
16 
16 
10 


0.5005      3260 
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D.   MODIFIED  MYOPIC  SOLUTION  HESOLTS 

The  reason  for  using  the  MMP  was  to  find  a  near  optimal 
starting  solution.  Ihis  objective  was  met  in  the  problems 
shown  so  far.  In  these  problems,  the  mean  position  of  the 
target  does  not  love,  given  no  search.  In  order  to  examine 
the  possible  effects  of  a  moving  mean  target  position,  two 
5x5  search  problems  were  solved  with  different  transition 
matrices,  initial  target  distributions,  and  detection 
probabilities. 

1 •   fast  Target  Problem 

In  this  problem  the  target  starts  in  cell  13  and  the 
searcher  starts  in  cell  1.  The  target  transition  is  such 
that  the  target  stays  in  the  previously  occupied  cell  with 
probability  0  and  moves  to  the  adjacent  cells  with  prob- 
ability 1/m  where  m  is  the  number  of  adjacent  cells.  The 
detecticn  probability  is  1.0  for  each  cell-  The  10-time 
period  search  results  are  shown  in  Table  5. 


TABLE  5 
5x5  FAST  TARGET  PROBLEM  RESULTS 


Search  Path 
Myopic   2  7  8   8  13  18  13  18  13 
k  =  1    2  2  7  12  17  18  19  14   9 
k  =  2    2  7  7  12  17  18  19  14   9 
k    =  3    2  7  7   8  13  12  17  18  19  14 
OPTIMAL  PND  =  0.3742 


PND 

CPU 
Time  {sec) 

8 

0.4770 

8 

0-3742 

1-9 

9 

0.3742 

10.  19 

4 

0.3779 

67.93 
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In  this  case  the  MMP  did  not  perform  as  well  as  it 
did  in  the  previous  5x5  problems.  An  optimal  solution  was 
obtained  for  k=1 ,  2.  Additionally,  k=3  policy  did  not  reach 
the    optimal  solution    while    requiring   more   CPU   time- 

2.      Escaping    Target   Problem 

For  this  problem,  the  target  starts  in  cell  21.  It 
then  stays  in  the  current  cell  with  probability  0.1  and 
moves  one  cell  up  or  to  the  right  with  probability  0.45.  If 
it  reaches  one  of  the  top  boundary  cells  (  1,  2,  3,  4  )  or 
one  of  the  right  boundary  cells  (  25,  20,  15,  10  ),  then  it 
will  move  one  cell  to  the  right  or  one  ceil  up  with  prob- 
ability 0.9  and  it  remains  in  the  current  cell  with  prob- 
ability 0.1-  When  it  reaches  cell  5  it  will  stay  there 
forever  and  is  assumed  to  have  escaped.  That  is,  the  prob- 
ability of  detection  in  this  cell  is  0,  while  for  the  otner 
cells  the  probability  of  detection  is  1.0.  The  U-time 
period  search  results  are  shown  in  Table  6.  The  ftMP  path 
has  significantly  greater  PND  than  the  optimal  path  even 
though  it  differs  in  only  one  time  period  from  an  optimal 
path.      Optimal  solutions    were    obtained   for    k=1,    2,    3. 

For  the  last  two  problems  all  optimal  paths  were 
obtained  by  total  enumeration  and  are  shown  in  Table  6.  In 
the  fast  target  problem,  the  optimal  search  path  placed 
search  effort  around  the  targets  starting  cell.  If  target 
moves  in  a  specific  direction  (escaping  target  problem)  this 
reduces  the  number  of  possible  optimal  paths.  In  this  case, 
optimal    paths    put    a    barrier    in    front    of    the    target. 

E.       INCEF.ASING   THE    HDHBEfi    OF    TIME    PEfilODS 

To  examine  the  effect  of  increasing  the  number  of  time 
periods  on  CPU  time,  3x3  problem  was  solved  for  15,  20,  25 
and    3C      time    periods.         The      graphical    results    are      shown    in 
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Myopic 

6 

k   =    1 

6 

k   =    2 

6 

k   =    3 

1 

TABLE   6 
ESCAPING    TARGET    PROBLEM    RESULTS 

Search  Path  PND         . CPU 

Ti me  /sec) 
11     12     12    13    14    15    15    10    10      0.2588 

11     11     12    13    14    15    15    10  10  0-1530  0-68 

11     11     12    13    14    15    15    10  10  0.1530  3.03 

6     11     12    13    14    15    15    10  10  0.1530  18.46 

OPTIMAL    PND    =    0.1530 


TOTAL    ENUMERATION    RESULTS    2- 

Optimal   CPU    lime 

Optimal  Paths  PND  (sec) 

9    14    19  18    17    12 

17    18    19  14      9       8 

9    14    19  18    17    12 

17    18    19  14      9       8 

9    14    19  18    17    12 

17    18    19  14       9      8  0.3742         1520 

9    14    19  18    17    12 

17    18    19  14      9       8 

9    14    19  18    17    12 

17    18    19  14      9       8 

9    14    19  18    17    12 

17    18    19  14      9       8 


6  6  11  12  13  14    15  15  10  10 
ESCA  PING 

TARGET         6  11  11  12  13  14    15  15  10  10       0.1530          1524 

1  6  11  12  13  14    15  15  10  10 


Problem 

Possibl 

6 

6 

7      8 

6 

6 

7    12 

6 

7 

7      8 

6 

7 

7     12 

FAST 
TARGET 

2 
2 

2 
2 

7      8 
7    12 

2 

7 

7      8 

2 

7 

7    12 

1 

2 

7      8 

1 

2 

7    12 

1 

6 

7      8 

1 

6 

7    12 
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Figure  4.5  for  k=1,  2,  and  3.  For  T=30  and  k=3  program  was 
stopped  at  3600  seconds  CPU  time.  The  increse  in  CPU  time 
for  k=1  was  considerably  less  than  for  k=2r  3.  we  can  say 
that  k=1  policy  was  the  most  CPU  TIME  /  (1-PND)  effective. 

Fcr  the  presented  problems  we  also  tested  the 
(k-1) -change  optimal  solution  as  a  starting  path  for  the 
k-change  policy  where  k=2.  But  starting  with  this  alterna- 
tive path  did  not  improve  the  solution  obtained  by  using  a 
MMP  starting  path. 
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V.     FBAHK-BOLFE    ALGOBITHM 

Another  method  applicable  to  the  problem  is  the 
Frank-Wolfe       (F-W)         algorithm.  The      main      motivation      of 

selecting  this  algorithm  is  that  this  method  implicitly 
examines  all  extreme  points  of  the  convex  polyhedron,  rather 
than    just   the   k-change  ones.  In    1956    Marguerite   Frank   and 

Philip  Wolfe  proposed  this  method  for  solving  non-linear 
programs  having  a  convex  diff erentiable  objective  function 
and  linear  constraints.  Since  our  objective  function  is  not 
convex  this  method  will  determine  an  approximate  solution  to 
the    problem. 

Ihe  F-W  algorithm  is  a  feasible  direction  method  and  may 
simply  stated  as  follows;  Given  a  feasible  point  at  any 
iteration,  an  improving  feasible  direction  is  determined  by 
linear    program    (LP).  The    objective   function    of      the   LP   is 

formed  by  using  a  linear  approximation  (first  order  Taylors 
expansion)  of  the  objective  function  of  the  NLP  and  the 
constraints  are  the  original  linear  constraints  of  the  NIP. 
Then  the  step  size  is  determined  by  a  line  search.  Ihe 
algorithm  continues  until  no  direction  is  found  to  improve 
the  objective  function.  For  further  information  refer  to 
[Bef.    8]. 

A.        AMPLICATION    OF    F-H    HETHOD 

Given  Sr  x  (t)  ,  P(t),  q(i)#  and  Z,  the  objective  function 
can    be    written  for  each    time   period    t   as    follows: 

PND(S)=^~    P(i,t)  {1-q(i)x<i,t)}z(i,j)v<j,t*1) 

i,j=1#2, ,H      and      t=1,2, ,T 
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where  v(i,t)  is  the  probability  that  the  target  is  in  cell  i 
at  the  beginning  of  time  period  t  and  will  not  be  detected 
by  the  remaining  searches.  He  can  compute  v(i,t)'s  in  a 
nackward  recursive  manner  as  fellows: 


v(i,T+1)=1- 0  ,   i=1#...,N 


v(i,t)=2 {1-g(i)x(i,t)}z(i,j)  v<j,t*1) 

xecii) 

i=1f 2f •• <<a  f N   and   t=1#2r«... r T 


Since  P  (1)   is   known,   the  p{i,t)  *s  can  be   computed  by  the 
forward  recursion 

P(jrt*1)  =  JT   P(i,t)  {1-q(i)x(i,t)}z(i,j) 
i 

i#  j=1,2#.««..  ,N   and   t=1,2, rT-1 


For  a  given  search  plan  S  the  p(i#t)  and  v(i,t)  will  be  the 
same  for  each  time  period.  So,  we  need  to  compute  them  once 
for  each  search  plan  S. 

Since  x(jrt)=     x  (i,  t-  1)  S  (i,  j,  t-1)  ,   then   the  gradient 
vector  for  time  period  t  can  be  computed  from  the  following: 

I(i'j't,=TsTr*^=~x(i't,p(j't*1)2_  q(j)z(j'k) v(k't+2)- 

W£CCj) 
i#j=1*2# ,N   and   t=  1,  2,  .„...  ,  1 


JO 


Then  the  feasible  direction  of  descent  is  found  by  solving 
the  IE  subproblero  shown  in  Table  7.  As  we  explained  before 
the  constraints  of  the  LP  must  be  the  same  as  the  NLP's 
constraints.  Note  that  the  Lf's  constraints  in  Tafcle  7  is 
another  way  of  expressing  the  constraints  in  Table  2.  The 
solution  of  this  LP  subproblems  is  trivially  found  as 
follows;  For  each  i  and  t,  choose  j  C (i)  witn  minimum 
Y(i,j,t),    call  it    j      then  set   M(i#j    ,t)  =  1   and   M(iFj,t)=0  for 

i  =  j  - 


" 

LP 

TABLE    7 
SOBPBOBLEM 

a  in 

5 

J 

(i,j,t)     M(i, 

jrt) 

subject 

to; 

J~Mi,3 
T£C(t) 

,t)=1 

t=l"" 

.,N 
.,T-1 

,       _   . .- 

j.t)3 

*0 

r               i=1,.- 
t=1,.. 

-rT-1 

This  solution  is  an  extreme  point  of  the  linear 
constraints.  Therefore,  solution  of  the  LP  is  deterministic 
search  plan  M  which  defines  a  unique  search  path  m.  Since 
we  are  only  interested  in  deterministic  search  paths,  we 
choose  the  step  size  as  1  if  PND{m)  <  PND(s)  and  zero  otner- 
wise,  ie.,  tne  process  is  terminated  when  PND(m)  >  PND(s). 
So,  the  second  algorithm  becomes: 

Step   1.     Select   an   arbitrary   feasible    search   path 
s={s(1),.-,s(T-1}. 
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Step  2.   Determine   a  new  search  path  m  by  solving  the  LP 
sur  problem. 

Step  3.    If  PND(m)    <  PND(s)    set  s=m,    go  to   step  1. 
Otherwise,  STOP. 


B.   CCMPUTATIOHAL  EXPEBIENCE  FOB  F-W  METHOD 

We  applied  the  algorithm  tc  the  3x3  problem  (defined  in 
Chapter  4)  with  30  randomly  selected  starting  search  paths 
(myopic  search  path  included).  The  algorithm  did  not  reach 
the  optimal  solution.  For  7  of  the  30  paths  (myopic  path 
was  one  of  them)  the  algorithm  did  not  improve  the  starting 
solution.  The  closest  solution  to  optimal  (0.2214)  was 
0.2328  which  is  the  myopic  path  result.  The  maximum  and 
minimum  CPU  times  were  0.04  and  0.11  seconds,  respectively. 
Ihe  results  are  shown  graphically  in  Figure  4.4.  The 
Fortran  code  of  the  algorithm  for  the  3x3  problem  is  given 
in  Appendix  B. 
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VI.    CONCIOSIONS 

We  have  investigated  the  effectiveness  and  computational 
efficiency  of  the  Local  Search  Method  and  the  Frank-Wolfe 
Method    for   several  two-dimensional   search   problems. 

The  heuristic  Local  Search  Method  appeared  clearly 
satisfactory  for  the  examined  search  problems.  However/  the 
exponential  increase  in  CPU  time  will  limit  the  use  of  the 
method  for  higher  k  values.  Also  increasing  k  does  not 
necessarily  improve  the  solution.  This  procedure  appears  to 
he  useful  because  it  allows  a  trade  off  between  solution 
accuracy,  i.e.,  closeness  to  the  optimal  search  path  and  CPU 
time.  For  further  analysis  this  method  can  be  tested  for 
different  problems,  k-change  policies  and  number  of  time 
periods. 

1  he  Frank-Wolfe  method  approach  did  not  perforc  well  for 
the       frobiems   examined.  An    alternative      way    of      improving 

efficiency  cf  this  algorithm  may  be  to  perform  a  line  search 
after  determining  the  feasible  direction  of  descent-  But 
this  would  increase  the  CPU  time.  Both  methods  are  also 
applicable  to  Brown's  original  problem  [ Bef .  9]  and  they  may 
be  efficient  for  this  problem  because  of  its  convex  objec- 
tive   function. 
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APPENDIX    A 
FORTRAN    CODE    OF    LOCAL    SEARCH    ALGORITHM 


C 
C 
C 
C 
C 
C 
C 
C 
C 
C 

c 

100 

103 

101 

102 

104 
C 
C 
C 

15 


25 
C 

c 
c 

35 


C 

C 

c 

999 
333 


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

X   METIN  SAGAL  * 

»   THIS  PROGRAM  APPLIES  LOCAL  SEARCH  * 

x   ALGORITHM  TO  3X3  PROBLFM  FOR  K=l.  * 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

INTEGER  T.NN,P(40  0,30),TT(50),S(50),C(25,25) 

INTEGER  AAC9.5)  , IN(9) 

REAL  PP(20),PB(50),BBC50),XMIN,PND 

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

x  INPUT  x 
T=TIME  PERIODS 
NN-tOF  SEARCH  CEL  LS 
S=STARTING  SEARCH  PATH 
1N=«0F  ADJACENT  CELLS 
A=ADJACENT  CEl LS 
C=CONSTRAINT  MATRIX 

XXXXX*XX*XXXXXXX*XXXXXXXXXXXXXXXXXXX*XXXXXXXXX 

READ(5,100)  T,NN 

F0RMATC3I2) 

READC  5,103)  (S( I ) , 1=1 ,T) 

F0RMAT( 1  111 ) 

READ(5.101)  (INC  I ) , 1  =  1 ,NN) 

FORMATC9I1 ) 

READC 5, 102)  AAC  I  ,  J  )  , J  =  l,5), I=1,NN) 

FORMATC5I1 ) 

RE ADC  5, 104)  CCCCI,J),J=1 rNN) ,1=1 ,NN) 

FORMATC9I2) 

xxxxx  TARGET  INITIAL  PROB.  DISTRIBUTION  xxxxxx 

DO  15  K=l ,NN 

PB(K)=0 
PB(9)=1 
DO  2  5  K=l ,NN 
BB(K)=PB(K) 

xxxxxxxxx  FIND  PND  FOR  STARTING  PATH  S  x*xxxxx 

DO  35  1=1.10 

Z( I )  =  S( 1  +  1  ) 
CALL  PROB(Z,PB,T,NN.BB,PND) 
XMIN=PND 

xxxxxxx*xx  FIND  NEIGHBORHOOD  PATHS  xxxxxxxxxxx 

MM=M=0 

TT  =  2 

K  =  IN(S(TT-D) 

DO  10  1=1 ,K 

ZCTT-1  )=AA(S(TT-1 ) , I ) 

xx*«**»xxx  CHECK  PATH  CONSTRAINTS  xxxxxxxxxxxx 
IF(C(Z(TT-1  )  ,S( TT+1  )  )  EQ. 0  )  GO  TO  10 

****x***xx  FIND  PND  FOR  NEW  PATH  »  »  *  x  *  «  «  *  *  x  x  x  x 
CALL  PKOB(Z,PB, T ,NN, BB.PHD) 


35 


30 


40 

10 

C 

C 

C 


50 


:oo 


72 
71 


M 
42 


IFCPND.LT .XMIN)  THEN  DO 
M  =  M  +  1 
MM=MM+1 

*********  REEF'  THE  BEST  PATH  AND  PND  ********* 

XMIN=PND 
DO  30  I B=l , 10 
P(M,IB)=Z(IB) 
M  =  0 
END  IF 

DO  40  IK=1 , 10 
Z(IK)=SCIK+1) 
CONTINUE 

xxxx*******  CHANGE  THE  NEXT  CELL  xxxxxxxxxxxxx 

TT=TT+1 

IFCTT.LE. 11)  GO  TO  333 

***  IF  THERE  IS  AN  IMPROVEMENT . THEN  SET  S  *<** 
*T0  THE  BEST  NEIGHBORHOOD  PATH  AND  START  AGAIN  * 

IF(MM.GT.O)  THEN  DO 
S(l)=l 

DO  50  1=2.11 

S(I  )  =  PC  1,1-1) 
Z( 1-1  )  =  P(1, 1-1  ) 
GO  TO  999 
END  IF 

*****  IF  THERE  IS  NO  IMPROVEMENT,  STOP  ******* 

PRINT,  '******  OPTIMAL  PATH  IS  ******* 
WR I  TEC b,20  0)  (SCI), 1=2. 11) 

FORMATC 1012) 

PRINT, 'OPTIMAL  PND  IS', XMIN 

STOP 

END 

**************  CALCULATES  PND  **************** 

SUBROUTINE  PROB(Z.PB.T.NN.BB.PND) 

REAL  A(50) ,PB( 50 ) ,PND, BBC50) 

INTEGER  NN,T,Z(50) 

KP=T-1 

DO  71  K=l ,KP 

PBCS(K)  )  =  0 

A(1)=.4*PB(1)+.2*PB(2)+.2*PB(4) 
A(2)=.3*PB(1)+.4*PB(2)+.3*PB(3)+.15*PB(5) 
A(3)=.?*PB(2)+.4*PB(3)+.2*PB(6) 
A(4)=.3*PB(l)+.4*PB(4)+.15*Pfl(5)+.3*PB(7) 
A(5)=.2*PB(2)+.2*PB(4)+.4*PB(5)+.2*Pb(6)+.2*PB(8) 
A(6)=.3*PB(3)+.15*PB(5)+.4*PB(6)+.3*PB(9) 
A(7)=.2*PB(4)+.4*PB(7)+.2*PB(S) 
A(8)=.15*PB(5)+.3*PB(7')+.4*PB(8)  +  .3*PB(9) 
A(9)=2*PB(6)+.2*PB(8)+.4*Pb(9) 
DO  72  IK=1.NN 
PB( IK  )=A( IK) 
CONTINUE 

com  I  rjUE 

P  B  (  5  (  T  )  )  =  0 

PND=0 . 0 

DO  51  K=l , NN 

PND=PND+Pb( K) 
CONTINUE 

DO  42  KM  ,  NN 
PB( K  )  =  BB( K) 
CONTINUE 
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RETURN 

ENE 

1 

C 

C     ************* 

C 

15  9 

12585658565 

343454343 

124 

1235 

236 

1457 

24568 

3569 

478 

5789 

689 

1  1  0 

1 

0 

0 

0 

0 

0 

1  1  1 

0 

1 

0 

0 

0 

0 

0  1  1 

0 

0 

1 

0 

0 

0 

1  0  0 

1 

1 

0 

1 

0 

0 

0  1  0 

1 

1 

1 

0 

1 

0 

0  0  1 

0 

1 

1 

0 

0 

1 

0  0  0 

1 

0 

0 

1 

1 

0 

0  0  0 

0 

1 

0 

1 

1 

1 

0  0  0 

0 

0 

1 

0 

1 

1 
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APPENDIX    B 
FOBTBAM   CODE    OF    F-H    ALGORI1H1 


INTEGER    TIME,T,N,INC9),AC9,9),S(10) , CELL.Z(ll) 
REAL     R(9,9),XC9.10),V(9,11),PND,BBC9),PBC9) 
REAL    CC9, 10)»XMIN,P(9, 10) 

C         xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

C  x    INPUT    * 

C      T=TIME  PERIODS 

C      N=»0F  CELLS 

C      S=STARTIHG  PATH 

C      IN=«0F  ADJACENT  CELLS 

C      A=ADJACENT  CELLS 

C      R=TRANSITIOH  MATRIX 

C    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

c 

READC5.100)  T,N 

100  F0RMATC2I2) 

READC5, 101 )  (SCI), 1=1, T) 

101  FORMAT(lOIl) 
READC5.102)  (  I  N( I  )  , I  =  1 , 9 ) 

102  F0RMATC9I1) 

READC5.103)  (  (  A  C  I  ,  J  )  .  J  =  1  ,  N  )  ,  I  =  1  ,  N  ) 

103  F0RMATC9I1) 

READC  5, 104)  ((R(I,J),J=1.N),I=1,N) 

104  F0RMATC9F3.2) 

xxxxxxxxxxxx  DETERMINE  V(I,J)'S  xxxxxxxxxxxxxx 

CALL  VFIND(S,T,N,V) 
xxxxxxxxxxxx  DETERMINE  P(I,J)'S  xxxxxxxxxxxxxx 
CALL  PFIND(S,T,P) 


DO  13  1=1 ,N 
PB(I)=P(I,1) 

13     BBC I)=P(  I,  1  ) 

xxxxxxxxxxxx  DETERMINE  X ( I , J ) ' S  xxxxxxxxxxxxxx 

CALL  XFINDC S, T,N,X) 

xxxxxxxx  FIND  PND  FOR  A  STARTING  PATH  **x*x*x* 

CAIL  PROB(S,PB,T,N, BB.PND) 
XXMIN=PND 

*xx*x**xx  FIND  NEW  EXTREME  POINT  x x x x x * x x x x x *  * 

ZC  1  )  =  1 
999   CALL  CFIND(2,Z(  1 ), IN.X.R, V,A,P,CELL  ) 

Z(2)=CELL 
CALL  CFIND(3,Z(2),IN,X,R.V.A,P,CELL) 

ZC  3)^CELL 
CALL  CFINDC3,ZC3),IN,X,R,V,A,P.CELL) 

Z(4)=CELL 
CALL  CFIND(4,Z(4).IN,X,R,V,A,P.CELL) 
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Z(5)=CELL 
CALL  CFIND(5,Z(5),IN,X,R,V,A,P,CELL) 

Z(6  )=CELL 
CALL  C  F  I  N  D  (  o  ,  Z  C  b  )  ,  I N  ,  X  .  R  .  V  .  A  ,  P  .  C  E  L  L  ) 

Z(7)^CELL 
CALL  CFIND(7.Z(7),IN,X,R.V,A,P,CELL) 

ZC8)=CELL 
CALL  CFIND(8,Z(8),IN,X,R,V,A,P,CELL) 

Z(9)=CELL 
CALL  CFIND(9,Z(9),IN,X,R,V,A.P,CELL) 

Z(10)=CELL 
CALL  CFIND(10,Z(10),IN,X,R,V,A,P,CELL) 

Z(11)=CELL 

xxxxxx  FIND  PND  FOR  A  NEW  EXTREME  POINT  ****** 

CALL  PROB(Z,PB,T,N,BB,PND) 

*******  COMPARE  TWO  EXTREME  POINT'S  PND  ****** 

IF(PND.LT.XXMIN)  THEN  DO 

XXMIN=PND 

DO  90  1  =  1, T 

5(1 )=Z(I  +  1  ) 

*  DETERMINE  NEW  V(I,J)'S,  P(I,J)'S,  X(I,J)'S  * 


11 


CALL  VFIND(5,T 
CALL  PFIND(S,T 
DO  11  1=1. N 
PBC I )=P( I . 1 ) 

BBC  I )=P(  I  , 1  ) 
CALL  XFINDCS,T 
GO  TO  999 
END  IF 


N,V) 
P) 


N,X) 


500 


600 


*x  IF  THE  NEW  EXTREME  POINT  DOES  NOT  IMPROVE  ** 
********  THE  OBJECTIVE  FUNCTION,  STOP  ********* 

PRINT, 'MIN  PND  IS' .XXMIN 

PRINT, '******  OPTIMAL  PATH  IS  ******' 

WRITEC6.6  00)  (5(1), 1=1, T) 

FORMATC3X. 1012) 
STOP 
END 

************  CALCULATES  V(I.J)'S  ************* 


15 
25 


SUBROUTINE  VFIND(S,T.N 
REAL  V(9, 11 ) 
INTEGER  S( 10)  ,T 
DO  15  I=1,N 

V( I ,  11  )  =  1  .0 


V) 


39 


55 


56 


«+0 


70 

60 

7  8 


END 

******  CALCULATES  COST  COEFFICIENTS  AND  ****** 
************  SOLVES  LP  SUBPKObLEM  ************ 


V,A,P,CELL  ) 
CMIN,X(9, 10  )  ,C(9»10) 
9) .CELL ,MI .MK(10),LL 


SUBROUTINE  CF1 ND( T I  ME > MM . I N , X , R . 
REAL  R(9,9) . V ( 9 . 11 ) ,P(9, 10)  .PP. 
INTEGER  TT I ME, TIME, MM, IN(9),A(9, 
CMIN=1 
I K  =  I N ( MM ) 
TTIME=TIME+1 
DO  40  1  =  1,  IK 
J=A(MM, I) 
PP=0.0 
I  B  =  I  N  (  J  ) 
IFC  PC  J, TIME) .EQ.O)  THEN  DO 
DO  55  11  =  1  ,IB 

PP=PP+R( J,A( J, II ) )*V(A( J 
CONTINUE 

C(MM, J)=-(0 . 0001 )*PP 
ELSE  DO 

DO  56  11  =  1  ,  IB 

PP=PP+R( J, A( J . II ) )*V(A( J 
CONTINUE 

C(MM, J)=-(P( J,T1ME)*PP) 
END  IF 
IFCCCMM. J) .LT.CMIN)  THEN  DO 
CMIN=C(MM, J) 
CELL=J 
END  IF 
CONTINUE 
RETURN 
END 


xxxxxxxxxxxxx  CALCULATES  X(I,J)'S  xxxxxxxxxxxx 


ID.TTIME) 


ID.TTIME) 


SUBROUTINE  XFI ND( S , T , N , X ) 
INTEGER  S(10),T 
REAL  XC9.10) 
DO  6  0  I=1,T 

DO  7  0  J  =  l  ,N 

X( J, I  )  =  0  .  0 

NTINUE 

NTINUE 

78  1=1, T 
X(S(  I  ),  I  )  =  1  .0 

URN 


CO 
CO 
DO 

RET 
END 


xx**x**xxx*x  CALCULATES  PND  x *xxx**x*xx* xxxx* x 


SUBROUTINE  PROBCS, 

REAL  AAC50)  ,PB( 9)  , 

INTEGER  N.T.S(IO) 

KP=T-1 

DO  7  1  K=1,KP 

PB(S(K)  )=0 

AA(  1  )=  .  4*PB(  1  )  + 


PB.T.N. BB.PND) 
PND, BB(9) 


2*PB( 
4  *  P  B  ( 
4*PB( 
4*PB( 
2*PR( 


AA(2)= . 3*PB( 1  )+  . 
AA(3)= .2*PB(2)+ . 
AA(4  )=  .  3*PB(  1  )+  . 
AA(5)= .2*PB(2)+  . 
AA(6  )= .3*PB(3)+. 
AA(7  )  =  .2*PB(4)+ . 
AA(8)= . 15*PB(5)+ 
AA(9)=.2*PB(6)+.2*PB(8)+ 

DO  7  2  IK=1.N 

PB( IK)=AA( IK) 

CONTINUE 


2*PB(4) 
3*PB(3)+ . 
2*PB(o  ) 
15*Pb( 5)+ 
4*PB( 5)+ . 
15*PBC5)+ 4*Pb(6  )  + 
4*PB( 7  )+ .2*PB(8) 
. 3*PB(7  )+ . 4*PB(8  )  + 
4*PB( 9  ) 


)  + 

)  i- 

)  + 

)  + 

)  + 


15*PB( 5) 

.  3  »  P  B  (  7  ) 
2*PB(6  )  + 
. 3*PB( 9  ) 

. 3*PB(9) 


2  *  P  B  (  8  ) 


40 


71    CONTINUE 

PBCSCT)  )  =  0 
PND=0.0 
DO  50  K-l  .N 

PND=PND+PB(K) 
50    CONTINUE 

DO  42  K  =  l  ,N 
PBCK)=BBCK) 
42    CONTINUE 
RETURN 
END 
C 

C    xxxxxxxxxxxx  CALCULATES  PCI,J)'S  xxxxxxxxxxxxx 
C 


85 


E  PFINDCS.T.P) 

10). PMC  9 ,10) 

C10),T,M 

0 

.0 

,8 

)  =  0 

=  0 

,T 


86 
80 


C 

C 

C 

10  9 

45658 

34345 

12400 

12350 

23600 

14570 

24  56  8 

35690 

4  7  8  0  0 

5/890 

68900 

04003 

02004 

0  0  0  0  3 

02000 

0  0  0  01 

0  0  0  0  0 

00000 

00000 

00000 


)=0.0 

.4*PM 
.3*PM 
.  Z»PM 
.3*PM 
.2<PM 
PC5,  J 
.3*PM 
.2*PM 
.  15*P 
.2*PM 
,9 
(I,  J) 


(1 
(  1 
(2 
CI 

(2 

)x 
(3 
(4 


M)+.2XPM(2 
M)+.4XPM(2 
M)+ .4XPM( 3 
M)+.4*PM(4 
M)+ . 2xPM(4 
2«PM(8,M) 
M)+. 15*PMC5,M)+ 
M)+ .4*PM(7 ,M)+  . 


M)  + 
M)  + 

M)  + 
M)  + 
M)  + 


2*PMC4,M) 

3*PM( 3,M)+. 15*PM(5,M) 

2*PM(6,M) 

15*PM(5.M)+.3*PMC7 ,N) 
4*PMC5.M)+ . 2* PMC  6 , M) 


3*PM(9,M) 


4 x PMC  6 ,  M)  + 
XPM(8,M) 

MC5,M)+.3*PMC7,M)+.4*PM(8.M)+.3*PM(9,M) 
C6,M)+.2*PMC8,M)+.4*PMC9,M) 


SUBROUTIN 
REAL  PC9, 
INTEGER  S 
PC9.  1  )  =  1  . 
PMC  9, 1)=1 
DO  85  1=1 
PMC  1,1 
PCI  ,  1  ) 
DO  80  J=2 
M=J-1 
PMC5CM) , M 

PC  1  ,  J)  = 

PC2, J)= 

PC3,  J)  = 

PC4,  J  )  = 

PC5 

PC5 

PC6 

PC7 

PC8 

PC9 
DO  86  1=1 
PMC  I, J  )=P 
CONTINUE 
RETURN 
END 


xxxxxxxxxxxxxxxxx  in PUT  DATA  xxxxxxxxxxxxxxxxx 


56585 
4343 
0000 
0000 
0000 
0  00  0 
0000 
0000 
0000 
0000 
0000 

0000030000 
00200  0  0020 
00400  0  00  00 
00  0  0040020 
50  0  0015040 
00200000  2  0 
00000  3  00  0  0 
0  00000  0  020 
00000  0  0  000 


0000 
0000 
0300 
0000 
0150 
0400 
0000 
0000 
0300 


0000 
0000 
0000 
2000 
0  001 
0  00  0 
4003 
2004 
0003 


0000 
0000 
0  0  00 
0000 
50  0  0 
0  020 
0000 
0  0  20 
0U40 
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